An advanced nomogram model using deep learning radiomics and clinical data for predicting occult lymph node metastasis in lung adenocarcinoma.

PURPOSE: To evaluate the effectiveness of deep learning radiomics nomogram in distinguishing the occult lymph node metastasis (OLNM) status in clinical stage IA lung adenocarcinoma. METHODS: A cohort of 473 cases of lung adenocarcinomas from two hospitals was included, with 404 cases allocated to the training cohort and 69 cases to the testing cohort. Clinical characteristics and semantic features were collected, and radiomics features were extracted from the computed tomography (CT) images. Additionally, deep transfer learning (DTL) features were generated using RseNet50. Predictive models were developed using the logistic regression (LR) machine learning algorithm. Moreover, gene analysis was conducted on RNA sequencing data from 14 patients to explore the underlying biological basis of deep learning radiomics scores. RESULT: The training and testing cohorts achieved AUC values of 0.826 and 0.775 for the clinical model, 0.865 and 0.801 for the radiomics model, 0.927 and 0.885 for the DTL-radiomics model, and 0.928 and 0.898 for the nomogram model. The nomogram model demonstrated superiority over the clinical model. The decision curve analysis (DCA) revealed a net benefit in predicting OLNM for all models. The investigation into the biological basis of deep learning radiomics scores identified an association between high scores and pathways related to tumor proliferation and immune cell infiltration in the microenvironment. CONCLUSIONS: The nomogram model, incorporating clinical-semantic features, radiomics, and DTL features, exhibited promising performance in predicting OLNM. It has the potential to provide valuable information for non-invasive lymph node staging and individualized therapeutic approaches.

Magnetically driven capsules with multimodal response and multifunctionality for biomedical applications.

Untethered capsules hold clinical potential for the diagnosis and treatment of gastrointestinal diseases. Although considerable progress has been achieved recently in this field, the constraints imposed by the narrow spatial structure of the capsule and complex gastrointestinal tract environment cause many open-ended problems, such as poor active motion and limited medical functions. In this work, we describe the development of small-scale magnetically driven capsules with a distinct magnetic soft valve made of dual-layer ferromagnetic soft composite films. A core technological advancement achieved is the flexible opening and closing of the magnetic soft valve by using the competitive interactions between magnetic gradient force and magnetic torque, laying the foundation for the functional integration of both drug release and sampling. Meanwhile, we propose a magnetic actuation strategy based on multi-frequency response control and demonstrate that it can achieve effective decoupled regulation of the capsule's global motion and local responses. Finally, through a comprehensive approach encompassing ideal models, animal ex vivo models, and in vivo assessment, we demonstrate the versatility of the developed magnetic capsules and their multiple potential applications in the biomedical field, such as targeted drug delivery and sampling, selective dual-drug release, and light/thermal-assisted therapy.

Combining external-latent attention for medical image segmentation.

The attention mechanism comes as a new entry point for improving the performance of medical image segmentation. How to reasonably assign weights is a key element of the attention mechanism, and the current popular schemes include the global squeezing and the non-local information interactions using self-attention (SA) operation. However, these approaches over-focus on external features and lack the exploitation of latent features. The global squeezing approach crudely represents the richness of contextual information by the global mean or maximum value, while non-local information interactions focus on the similarity of external features between different regions. Both ignore the fact that the contextual information is presented more in terms of the latent features like the frequency change within the data. To tackle above problems and make proper use of attention mechanisms in medical image segmentation, we propose an external-latent attention collaborative guided image segmentation network, named TransGuider. This network consists of three key components: 1) a latent attention module that uses an improved entropy quantification method to accurately explore and locate the distribution of latent contextual information. 2) an external self-attention module using sparse representation, which can preserve external global contextual information while reducing computational overhead by selecting representative feature description map for SA operation. 3) a multi-attention collaborative module to guide the network to continuously focus on the region of interest, refining the segmentation mask. Our experimental results on several benchmark medical image segmentation datasets show that TransGuider outperforms the state-of-the-art methods, and extensive ablation experiments demonstrate the effectiveness of the proposed components. Our code will be available at https://github.com/chasingone/TransGuider.

Development and Validation of a Deep Learning Radiomics Model to Predict High-Risk Pathologic Pulmonary Nodules Using Preoperative Computed Tomography.

RATIONALE AND OBJECTIVES: To accurately identify the high-risk pathological factors of pulmonary nodules, our study constructed a model combined with clinical features, radiomics features, and deep transfer learning features to predict high-risk pathological pulmonary nodules. MATERIALS AND METHODS: The study cohort consisted of 469 cases of lung adenocarcinoma patients, divided into a training cohort (n = 400) and an external validation cohort (n = 69). We obtained computed tomography (CT) semantic features and clinical characteristics, as well as extracted radiomics and deep transfer learning (DTL) features from the CT images. Selected features were used for constructing prediction models using the logistic regression (LR) algorithm. The performance of the models was evaluated through metrics including the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, calibration curve, and decision curve analysis. RESULTS: The clinical model achieved an AUC of 0.774 (95% CI: 0.728-0.821) in the training cohort and 0.762 (95% confidence interval [CI]: 0.650-0.873) in the external validation cohort. The radiomics model demonstrated an AUC of 0.847 (95% CI: 0.810-0.884) in the training cohort and 0.800 (95% CI: 0.693-0.907) in the external validation cohort. The radiomics-DTL (Rad-DTL) model showed an AUC of 0.871 (95% CI: 0.838-0.905) in the training cohort and 0.806 (95% CI: 0.698-0.914) in the external validation cohort. The proposed combined model yielded AUC values of 0.872 and 0.814 in the training and external validation cohorts, respectively. The combined model demonstrated superiority over both the clinical model and the Rad-DTL model. There were no statistically significant differences observed in the comparison between the combined model incorporating clinical features and the Rad-DTL model. Decision curve analysis (DCA) indicated that the models provided a net benefit in predicting high-risk pathologic pulmonary nodules. CONCLUSION: Rad-DTL signature is a potential biomarker for predicting high-risk pathologic pulmonary nodules using preoperative CT, determining the appropriate surgical strategy, and guiding the extent of resection.

FSA-Net: Rethinking the attention mechanisms in medical image segmentation from releasing global suppressed information.

Attention mechanism-based medical image segmentation methods have developed rapidly recently. For the attention mechanisms, it is crucial to accurately capture the distribution weights of the effective features contained in the data. To accomplish this task, most attention mechanisms prefer using the global squeezing approach. However, it will lead to a problem of over-focusing on the global most salient effective features of the region of interest, while suppressing the secondary salient ones. Making partial fine-grained features are abandoned directly. To address this issue, we propose to use a multiple-local perception method to aggregate global effective features, and design a fine-grained medical image segmentation network, named FSA-Net. This network consists of two key components: 1) the novel Separable Attention Mechanisms which replace global squeezing with local squeezing to release the suppressed secondary salient effective features. 2) a Multi-Attention Aggregator (MAA) which can fuse multi-level attention to efficiently aggregate task-relevant semantic information. We conduct extensive experimental evaluations on five publicly available medical image segmentation datasets: MoNuSeg, COVID-19-CT100, GlaS, CVC-ClinicDB, ISIC2018, and DRIVE datasets. Experimental results show that FSA-Net outperforms state-of-the-art methods in medical image segmentation.

A bidirectional registration neural network for cardiac motion tracking using cine MRI images.

Using cine magnetic resonance imaging (cine MRI) images to track cardiac motion helps users to analyze the myocardial strain, and is of great importance in clinical applications. At present, most of the automatic deep learning-based motion tracking methods compare two images without considering temporal information between MRI frames, which easily leads to the lack of consistency of the generated motion fields. Even though a small number of works take into account the temporal factor, they are usually computationally intensive or have limitations on image length. To solve this problem, we propose a bidirectional convolution neural network for motion tracking of cardiac cine MRI images. This network leverages convolutional blocks to extract spatial features from three-dimensional (3D) image registration pairs, and models the temporal relations through a bidirectional recurrent neural network to obtain the Lagrange motion field between the reference image and other images. Compared with previous pairwise registration methods, the proposed method can automatically learn spatiotemporal information from multiple images with fewer parameters. We evaluated our model on three public cardiac cine MRI datasets. The experimental results demonstrated that the proposed method can significantly improve the motion tracking accuracy. The average Dice coefficient between estimated segmentation and manual segmentation has reached almost 0.85 on the widely used Automatic Cardiac Diagnostic Challenge (ACDC) dataset.

A discontinuity-preserving regularization for deep learning-based cardiac image registration.

Objective. Sliding motion may occur between organs in anatomical regions due to respiratory motion and heart beating. This issue is often neglected in previous studies, resulting in poor image registration performance. A new approach is proposed to handle discontinuity at the boundary and improve registration accuracy.Approach. The proposed discontinuity-preserving regularization (DPR) term can maintain local discontinuities. It leverages the segmentation mask to find organ boundaries and then relaxes the displacement field constraints in these boundary regions. A weakly supervised method using mask dissimilarity loss (MDL) is also proposed. It employs a simple formula to calculate the similarity between the fixed image mask and the deformed moving image mask. These two strategies are added to the loss function during network training to guide the model better to update parameters. Furthermore, during inference time, no segmentation mask information is needed.Main results. Adding the proposed DPR term increases the Dice coefficients by 0.005, 0.009, and 0.081 for three existing registration neural networks CRNet, VoxelMorph, and ViT-V-Net, respectively. It also shows significant improvements in other metrics, including Hausdorff Distance and Average Surface Distance. All quantitative indicator results with MDL have been slightly improved within 1%. After applying these two regularization terms, the generated displacement field is more reasonable at the boundary, and the deformed moving image is closer to the fixed image.Significance. This study demonstrates that the proposed regularization terms can effectively handle discontinuities at the boundaries of organs and improve the accuracy of deep learning-based cardiac image registration methods. Besides, they are generic to be extended to other networks.

A modality-collaborative convolution and transformer hybrid network for unpaired multi-modal medical image segmentation with limited annotations.

BACKGROUND: Multi-modal learning is widely adopted to learn the latent complementary information between different modalities in multi-modal medical image segmentation tasks. Nevertheless, the traditional multi-modal learning methods require spatially well-aligned and paired multi-modal images for supervised training, which cannot leverage unpaired multi-modal images with spatial misalignment and modality discrepancy. For training accurate multi-modal segmentation networks using easily accessible and low-cost unpaired multi-modal images in clinical practice, unpaired multi-modal learning has received comprehensive attention recently. PURPOSE: Existing unpaired multi-modal learning methods usually focus on the intensity distribution gap but ignore the scale variation problem between different modalities. Besides, within existing methods, shared convolutional kernels are frequently employed to capture common patterns in all modalities, but they are typically inefficient at learning global contextual information. On the other hand, existing methods highly rely on a large number of labeled unpaired multi-modal scans for training, which ignores the practical scenario when labeled data is limited. To solve the above problems, we propose a modality-collaborative convolution and transformer hybrid network (MCTHNet) using semi-supervised learning for unpaired multi-modal segmentation with limited annotations, which not only collaboratively learns modality-specific and modality-invariant representations, but also could automatically leverage extensive unlabeled scans for improving performance. METHODS: We make three main contributions to the proposed method. First, to alleviate the intensity distribution gap and scale variation problems across modalities, we develop a modality-specific scale-aware convolution (MSSC) module that can adaptively adjust the receptive field sizes and feature normalization parameters according to the input. Secondly, we propose a modality-invariant vision transformer (MIViT) module as the shared bottleneck layer for all modalities, which implicitly incorporates convolution-like local operations with the global processing of transformers for learning generalizable modality-invariant representations. Third, we design a multi-modal cross pseudo supervision (MCPS) method for semi-supervised learning, which enforces the consistency between the pseudo segmentation maps generated by two perturbed networks to acquire abundant annotation information from unlabeled unpaired multi-modal scans. RESULTS: Extensive experiments are performed on two unpaired CT and MR segmentation datasets, including a cardiac substructure dataset derived from the MMWHS-2017 dataset and an abdominal multi-organ dataset consisting of the BTCV and CHAOS datasets. Experiment results show that our proposed method significantly outperforms other existing state-of-the-art methods under various labeling ratios, and achieves a comparable segmentation performance close to single-modal methods with fully labeled data by only leveraging a small portion of labeled data. Specifically, when the labeling ratio is 25%, our proposed method achieves overall mean DSC values of 78.56% and 76.18% in cardiac and abdominal segmentation, respectively, which significantly improves the average DSC value of two tasks by 12.84% compared to single-modal U-Net models. CONCLUSIONS: Our proposed method is beneficial for reducing the annotation burden of unpaired multi-modal medical images in clinical applications.

Prostate lesion segmentation based on a 3D end-to-end convolution neural network with deep multi-scale attention.

Prostate cancer is one of the deadest cancers among human beings. To better diagnose the prostate cancer, prostate lesion segmentation becomes a very important work, but its progress is very slow due to the prostate lesions small in size, irregular in shape, and blurred in contour. Therefore, automatic prostate lesion segmentation from mp-MRI is a great significant work and a challenging task. However, the most existing multi-step segmentation methods based on voxel-level classification are time-consuming, may introduce errors in different steps and lead to error accumulation. To decrease the computation time, harness richer 3D spatial features, and fuse the multi-level contextual information of mp-MRI, we present an automatic segmentation method in which all steps are optimized conjointly as one step to form our end-to-end convolutional neural network. The proposed end-to-end network DMSA-V-Net consists of two parts: (1) a 3D V-Net is used as the backbone network, it is the first attempt in employing 3D convolutional neural network for CS prostate lesion segmentation, (2) a deep multi-scale attention mechanism is introduced into the 3D V-Net which can highly focus on the ROI while suppressing the redundant background. As a merit, the attention can adaptively re-align the context information between the feature maps at different scales and the saliency maps in high-levels. We performed experiments based on five cross-fold validation with data including 97 patients. The results show that the Dice and sensitivity are 0.7014 and 0.8652 respectively, which demonstrates that our segmentation approach is more significant and accurate compared to other methods.

Automatic 3D CT liver segmentation based on fast global minimization of probabilistic active contour.

PURPOSE: Automatic liver segmentation from computed tomography (CT) images is an essential preprocessing step for computer-aided diagnosis of liver diseases. However, due to the large differences in liver shapes, low-contrast to adjacent tissues, and existence of tumors or other abnormalities, liver segmentation has been very challenging. This study presents an accurate and fast liver segmentation method based on a novel probabilistic active contour (PAC) model and its fast global minimization scheme (3D-FGMPAC), which is explainable as compared with deep learning methods. METHODS: The proposed method first constructs a slice-indexed-histogram to localize the volume of interest (VOI) and estimate the probability that a voxel belongs to the liver according its intensity. The probabilistic image would be used to initialize the 3D PAC model. Secondly, a new contour indicator function, which is a component of the model, is produced by combining the gradient-based edge detection and Hessian-matrix-based surface detection. Then, a fast numerical scheme derived for the 3D PAC model is performed to evolve the initial probabilistic image into the global minimizer of the model, which is a smoothed probabilistic image showing a distinctly highlighted liver. Next, a simple region-growing strategy is applied to extract the whole liver mask from the smoothed probabilistic image. Finally, a B-spline surface is constructed to fit the patch of the rib cage to prevent possible leakage into adjacent intercostal tissues. RESULTS: The proposed method is evaluated on two public datasets. The average Dice score, volume overlap error, volume difference, symmetric surface distance and volume processing time are 0.96, 7.35%, 0.02%, 1.17 mm and 19.8 s for the Sliver07 dataset, and 0.95, 8.89%, - 0.02 % $-0.02\%$ , 1.45 mm and 23.08 s for the 3Dircadb dataset, respectively. CONCLUSIONS: The proposed fully-automatic approach can effectively segment the liver from low-contrast and complex backgrounds. The quantitative and qualitative results demonstrate that the proposed segmentation method outperforms state-of-the-art traditional automatic liver segmentation algorithms and achieves very competitive performance compared with recent deep leaning-based methods.

A 3D Cross-Modality Feature Interaction Network With Volumetric Feature Alignment for Brain Tumor and Tissue Segmentation.

Accurate volumetric segmentation of brain tumors and tissues is beneficial for quantitative brain analysis and brain disease identification in multi-modal Magnetic Resonance (MR) images. Nevertheless, due to the complex relationship between modalities, 3D Fully Convolutional Networks (3D FCNs) using simple multi-modal fusion strategies hardly learn the complex and nonlinear complementary information between modalities. Meanwhile, the indiscriminative feature aggregation between low-level and high-level features easily causes volumetric feature misalignment in 3D FCNs. On the other hand, the 3D convolution operations of 3D FCNs are excellent at modeling local relations but typically inefficient at capturing global relations between distant regions in volumetric images. To tackle these issues, we propose an Aligned Cross-Modality Interaction Network (ACMINet) for segmenting the regions of brain tumors and tissues from MR images. In this network, the cross-modality feature interaction module is first designed to adaptively and efficiently fuse and refine multi-modal features. Secondly, the volumetric feature alignment module is developed for dynamically aligning low-level and high-level features by the learnable volumetric feature deformation field. Thirdly, we propose the volumetric dual interaction graph reasoning module for graph-based global context modeling in spatial and channel dimensions. Our proposed method is applied to brain glioma, vestibular schwannoma, and brain tissue segmentation tasks, and we performed extensive experiments on BraTS2018, BraTS2020, Vestibular Schwannoma, and iSeg-2017 datasets. Experimental results show that ACMINet achieves state-of-the-art segmentation performance on all four benchmark datasets and obtains the highest DSC score of hard-segmented enhanced tumor region on the validation leaderboard of the BraTS2020 challenge.

A bidirectional multilayer contrastive adaptation network with anatomical structure preservation for unpaired cross-modality medical image segmentation.

Multi-modal medical image segmentation has achieved great success through supervised deep learning networks. However, because of domain shift and limited annotation information, unpaired cross-modality segmentation tasks are still challenging. The unsupervised domain adaptation (UDA) methods can alleviate the segmentation degradation of cross-modality segmentation by knowledge transfer between different domains, but current methods still suffer from the problems of model collapse, adversarial training instability, and mismatch of anatomical structures. To tackle these issues, we propose a bidirectional multilayer contrastive adaptation network (BMCAN) for unpaired cross-modality segmentation. The shared encoder is first adopted for learning modality-invariant encoding representations in image synthesis and segmentation simultaneously. Secondly, to retain the anatomical structure consistency in cross-modality image synthesis, we present a structure-constrained cross-modality image translation approach for image alignment. Thirdly, we construct a bidirectional multilayer contrastive learning approach to preserve the anatomical structures and enhance encoding representations, which utilizes two groups of domain-specific multilayer perceptron (MLP) networks to learn modality-specific features. Finally, a semantic information adversarial learning approach is designed to learn structural similarities of semantic outputs for output space alignment. Our proposed method was tested on three different cross-modality segmentation tasks: brain tissue, brain tumor, and cardiac substructure segmentation. Compared with other UDA methods, experimental results show that our proposed BMCAN achieves state-of-the-art segmentation performance on the above three tasks, and it has fewer training components and better feature representations for overcoming overfitting and domain shift problems. Our proposed method can efficiently reduce the annotation burden of radiologists in cross-modality image analysis.

Automatic CT liver Couinaud segmentation based on key bifurcation detection with attentive residual hourglass-based cascaded network.

This paper presents an automatic Couinaud segmentation method based on deep learning of key point detection. Assuming that the liver mask has been extracted, the proposed method can automatically divide the liver into eight anatomical segments according to Couinaud's definition. Firstly, an attentive residual hourglass-based cascaded network (ARH-CNet) is proposed to identify six key bifurcation points of the hepatic vascular system. Subsequently, the detected points are used to derive the planes that divide the liver into different functional units, and the caudate lobe is segmented slice-by-slice based on the circles defined by the detected points. We comprehensively evaluate our method on a public dataset from MICCAI 2018. Experiments firstly demonstrate the effectiveness of our landmark detection network ARH-CNet, which is superior to that of two baseline methods, also robust to noisy data. The average error distance of all predicted key points is 4.68 ± 3.17 mm, and the average accuracy of all points is 90% with the detection error distance of 7 mm. We also verify that summation of the corresponding heat-maps can improve the accuracy of point localization. Furthermore, the overlap-based accuracy and the Dice score of our landmark-derived Couinaud segmentation are respectively 91% and 84%, which are better than the performance of the direct segmentation approach and the traditional plane-based method, thus our method can be regarded as a good alternative for automatic Couinaud segmentation.

APRNet: A 3D Anisotropic Pyramidal Reversible Network With Multi-Modal Cross-Dimension Attention for Brain Tissue Segmentation in MR Images.

Brain tissue segmentation in multi-modal magnetic resonance (MR) images is significant for the clinical diagnosis of brain diseases. Due to blurred boundaries, low contrast, and intricate anatomical relationships between brain tissue regions, automatic brain tissue segmentation without prior knowledge is still challenging. This paper presents a novel 3D fully convolutional network (FCN) for brain tissue segmentation, called APRNet. In this network, we first propose a 3D anisotropic pyramidal convolutional reversible residual sequence (3DAPC-RRS) module to integrate the intra-slice information with the inter-slice information without significant memory consumption; secondly, we design a multi-modal cross-dimension attention (MCDA) module to automatically capture the effective information in each dimension of multi-modal images; then, we apply 3DAPC-RRS modules and MCDA modules to a 3D FCN with multiple encoded streams and one decoded stream for constituting the overall architecture of APRNet. We evaluated APRNet on two benchmark challenges, namely MRBrainS13 and iSeg-2017. The experimental results show that APRNet yields state-of-the-art segmentation results on both benchmark challenge datasets and achieves the best segmentation performance on the cerebrospinal fluid region. Compared with other methods, our proposed approach exploits the complementary information of different modalities to segment brain tissue regions in both adult and infant MR images, and it achieves the average Dice coefficient of 87.22% and 93.03% on the MRBrainS13 and iSeg-2017 testing data, respectively. The proposed method is beneficial for quantitative brain analysis in the clinical study, and our code is made publicly available.

Lung-CRNet: A convolutional recurrent neural network for lung 4DCT image registration.

PURPOSE: Deformable image registration (DIR) of lung four-dimensional computed tomography (4DCT) plays a vital role in a wide range of clinical applications. Most of the existing deep learning-based lung 4DCT DIR methods focus on pairwise registration which aims to register two images with large deformation. However, the temporal continuities of deformation fields between phases are ignored. This paper proposes a fast and accurate deep learning-based lung 4DCT DIR approach that leverages the temporal component of 4DCT images. METHODS: We present Lung-CRNet, an end-to-end convolutional recurrent registration neural network for lung 4DCT images and reformulate 4DCT DIR as a spatiotemporal sequence predicting problem in which the input is a sequence of three-dimensional computed tomography images from the inspiratory phase to the expiratory phase in a respiratory cycle. The first phase in the sequence is selected as the only reference image and the rest as moving images. Multiple convolutional gated recurrent units (ConvGRUs) are stacked to capture the temporal clues between images. The proposed network is trained in an unsupervised way using a spatial transformer layer. During inference, Lung-CRNet is able to yield the respective displacement field for each reference-moving image pair in the input sequence. RESULTS: We have trained the proposed network using a publicly available lung 4DCT dataset and evaluated performance on the widely used the DIR-Lab dataset. The mean and standard deviation of target registration error are 1.56 ± 1.05 mm on the DIR-Lab dataset. The computation time for each forward prediction is less than 1 s on average. CONCLUSIONS: The proposed Lung-CRNet is comparable to the existing state-of-the-art deep learning-based 4DCT DIR methods in both accuracy and speed. Additionally, the architecture of Lung-CRNet can be generalized to suit other groupwise registration tasks which align multiple images simultaneously.

Learning U-Net Based Multi-Scale Features in Encoding-Decoding for MR Image Brain Tissue Segmentation.

Accurate brain tissue segmentation of MRI is vital to diagnosis aiding, treatment planning, and neurologic condition monitoring. As an excellent convolutional neural network (CNN), U-Net is widely used in MR image segmentation as it usually generates high-precision features. However, the performance of U-Net is considerably restricted due to the variable shapes of the segmented targets in MRI and the information loss of down-sampling and up-sampling operations. Therefore, we propose a novel network by introducing spatial and channel dimensions-based multi-scale feature information extractors into its encoding-decoding framework, which is helpful in extracting rich multi-scale features while highlighting the details of higher-level features in the encoding part, and recovering the corresponding localization to a higher resolution layer in the decoding part. Concretely, we propose two information extractors, multi-branch pooling, called MP, in the encoding part, and multi-branch dense prediction, called MDP, in the decoding part, to extract multi-scale features. Additionally, we designed a new multi-branch output structure with MDP in the decoding part to form more accurate edge-preserving predicting maps by integrating the dense adjacent prediction features at different scales. Finally, the proposed method is tested on datasets MRbrainS13, IBSR18, and ISeg2017. We find that the proposed network performs higher accuracy in segmenting MRI brain tissues and it is better than the leading method of 2018 at the segmentation of GM and CSF. Therefore, it can be a useful tool for diagnostic applications, such as brain MRI segmentation and diagnosing.

An Explainable System for Diagnosis and Prognosis of COVID-19.

The outbreak of Coronavirus Disease-2019 (COVID-19) has posed a threat to world health. With the increasing number of people infected, healthcare systems, especially those in developing countries, are bearing tremendous pressure. There is an urgent need for the diagnosis of COVID-19 and the prognosis of inpatients. To alleviate these problems, a data-driven medical assistance system is put forward in this article. Based on two real-world data sets in Wuhan, China, the proposed system integrates data from different sources with tools of machine learning (ML) to predict COVID-19 infected probability of suspected patients in their first visit, and then predict mortality of confirmed cases. Rather than choosing an interpretable algorithm, this system separates the explanations from ML models. It can do help to patient triaging and provide some useful advice for doctors.

A new data augmentation method based on local image warping for medical image segmentation.

PURPOSE: The segmentation accuracy of medical images was improved by increasing the number of training samples using a local image warping technique. The performance of the proposed method was evaluated in the segmentation of breast masses, prostate and brain tumors, and lung nodules. METHODS: We propose a simple data augmentation method which is called stochastic evolution (SE). Specifically, the idea of SE stems from our thinking about the deterioration of the diseased tissue and the healing process. In order to simulate this natural process, we implement it according to the local distortion algorithm in image warping. In other words, the irregular deterioration and healing processes of the diseased tissue is simulated according to the direction of the local distortion, thereby producing a natural sample that is indistinguishable by humans. RESULTS: The proposed method is evaluated on four segmentation tasks of breast masses, prostate, brain tumors, and lung nodules. Comparing the experimental results of four segmentation methods based on the UNet segmentation architecture without adding any expanded data during training, the accuracy and the Hausdorff distance obtained in our approach remain almost the same as other methods. However, the dice similarity coefficient (DSC) and sensitivity (SEN) have both improved to some extent. Among them, DSC is increased by 5.2%, 2.8%, 1.0%, and 3.2%, respectively; SEN is increased by 6.9%, 4.3%, 1.2%, and 4.5%, respectively. CONCLUSIONS: Experimental results show that the proposed SE data augmentation method could improve the segmentation accuracy of breast masses, prostate, brain tumors, and lung nodules. The method also shows the robustness with different image datasets and imaging modalities.

Multi-model Ensemble Learning Architecture Based on 3D CNN for Lung Nodule Malignancy Suspiciousness Classification.

Classification of benign and malignant in lung nodules using chest CT images is a key step in the diagnosis of early-stage lung cancer, as well as an effective way to improve the patients' survival rate. However, due to the diversity of lung nodules and the visual similarity of lung nodules to their surrounding tissues, it is difficult to construct a robust classification model with conventional deep learning-based diagnostic methods. To address this problem, we propose a multi-model ensemble learning architecture based on 3D convolutional neural network (MMEL-3DCNN). This approach incorporates three key ideas: (1) Constructed multi-model network architecture can be well adapted to the heterogeneity of lung nodules. (2) The input that concatenated of the intensity image corresponding to the nodule mask, the original image, and the enhanced image corresponding to which can help training model to extract advanced feature with more discriminative capacity. (3) Select the corresponding model to different nodule size dynamically for prediction, which can improve the generalization ability of the model effectively. In addition, ensemble learning is applied in this paper to further improve the robustness of the nodule classification model. The proposed method has been experimentally verified on the public dataset, LIDC-IDRI. The experimental results show that the proposed MMEL-3DCNN architecture can obtain satisfactory classification results.

A Two-Stage Convolutional Neural Networks for Lung Nodule Detection.

Early detection of lung cancer is an effective way to improve the survival rate of patients. It is a critical step to have accurate detection of lung nodules in computed tomography (CT) images for the diagnosis of lung cancer. However, due to the heterogeneity of the lung nodules and the complexity of the surrounding environment, it is a challenge to develop a robust nodule detection method. In this study, we propose a two-stage convolutional neural networks (TSCNN) for lung nodule detection. The first stage based on the improved U-Net segmentation network is to establish an initial detection of lung nodules. During this stage, in order to obtain a high recall rate without introducing excessive false positive nodules, we propose a new sampling strategy for training. Simultaneously, a two-phase prediction method is also proposed in this stage. The second stage in the TSCNN architecture based on the proposed dual pooling structure is built into three 3D-CNN classification networks for false positive reduction. Since the network training requires a significant amount of training data, we designed a random mask as the data augmentation method in this study. Furthermore, we have improved the generalization ability of the false positive reduction model by means of ensemble learning. We verified the proposed architecture on the LUNA dataset in our experiments, which showed that the proposed TSCNN architecture did obtain competitive detection performance.